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ABSTRACT 

Educational Resources Information Center (ERIC), the 
Defense Documentation Center (DDC) , National Technical Information 
Service (NTIS) , Psychological Abstracts, HumRRO Library, and 
industrial training publications were surveyed to analyze 
instructional and evaluative techniques relevant to team training. 
Research studies and team training practices underway within the 
military were also examined. From this review was developed a 
classification scheme for team training and evaluation to be used by 
the Defense Advanced Research Projects Agency (DARPA) . Training 
situations were caregorized as emergent or established and team 
training was distinguished from multi-individual training. Research 
needs in team training were identified using this classification 
scheme. Ninety-seven references on training and evaluation published 
between 1952 and 1976 are provided. (CH) 
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SUMMARY 

BACKGROUND 

This review of team training and evaluation was undertaken by HumRRO in 
order to provide an information base that could be used by DARPA as a foundation 
to facilitate decisions regarding future research program support. In order 
to pursue their mission of operational readiness, the Services conduct most of 
their training in the operational commands. However, in the past, most training 
research has focused on individual training at schools and at other institutional 
locations. Considering the amount of team training that goes on in the Services, 
either formally recognized as training or combined with operations, the funds 
committed to R&D support are relatively small. There is a need, therefore, for 
increased research emphasis on team training and evaluation. 

PURPOSE 

The purpose of this literature review was to provide information to DARPA 
which j^ould be useful for planning research and development programs in the area 
of team training. Such programs are needed to develop improved team training 
methods and technologies, as well as to provide new measurement techniques and 
procedures for evaluation. The specific objectives in preparing this review 
were to address the following questions: 

• VThat state-of-art gaps are there in team training strategies and 
evaluation techniques? 

• What new team training strategies appear to hold promise for 
application to the DoD environment? 

• What new evaluation techniques can be used to assess team training? 



METHOD 

A thorough search was made of the major documentation sources for publica- 
tions which were relevant to team training/evaluation* Initial sources searched 
included the Educational Resources Information Center (ERIC) , National Technical 
Information Service (NTIS) , and Defense Documentation Center (DDC) indices • In 
addition to DoD-sponsored research reports, publications from the social 
psychological areas such as group dynamics and the industrial training field 
were surveyed. Secondary sources and cross-references were then identified and 
reviewed, HumRRO personnel with experience in the team training area were 
consulted for information about current military team training methods. The most 
recent militar>' doctrine regarding team training procedures was obtained and 
reviewed. Military personnel iwere contacted in all the Serrices to identify team 
training approaches now in operation. Current team training and evaluation 
studies underway within the Army Research Institute and the other Services were 
examined and described, 

FINDINGS AND IMPLICATIONS 

A classification scheme was introduced in which the training techniques and 
situations discussed in this review were categorized along two dimensions. On 
one dimension, a distinction was made between "team" training and "multi-individual" 
training. Multi-individual training occurs in a group context but focuses on the 
development of individual skills. On the other hand, the focus of team training 
is on developing team skills (e.g, , co rdination, cooperation, etc.). The type 
of task situation was the other dimension used to classify the training techniques 
reviewed in this study.* Task situations were categorized as either "established" 
or "emergent," Established situations are those in which the tasks and the 
activities required to perform these tasks can be almost completely specified. 

4 

O 2 

ERIC 



Emergent situations are t,iose in which all tasks and activities cannot be specified 
and the probable consequences of certain actions cannot be predicted. This type 
of situation allows for unanticipated behaviors to emerge. 

Team training studies and practices were categorized according to the 
classification scheme described above. These studies were described in this 
review as having followed two conceptual models of team behavior — Stimulus- 
Response (S-R) and Organismic. The S-R model adherents tended to study team 
training in laboratory settings derived from established task situations. More 
realistic environments were used by other researchers who attended to emergent 
factors in the job situation (the organismic approach). It was this latter 
group of investigators who demonstrated the need for training in team skills, even 
though individual skill proficiency was found to be a prerequisite for effective 
team training and performance. Other conclusions which were drawn from the 
literature are: 

• The team context is not the proper location for initial individual 
skill acquisition. 

• Performance feedback is critical to the learning of team as well as 
individual skills. 

Current military team training practices were described as varying in cost, 
in degree of fidelity to the operational site, and in the variety and complexity 
of the tasks trained* Subjective methods of evaluation were found to be pre- 
dominant. 

Evaluation of team training effectiveness depends upon the development of 
objective team performance measurement instruments and procedures ."^--^^This review 
of the literature and current practice support the conclusion that this area -..^ 
contains many problems which must be solved before substantial improvement in 
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team training evaluation can occur. The production of standardizable, replicable 
test conditions and the establishment of accepted performance criteria are two 
such problems • 

A phased research program was suggested as warranting support in order to 
overcome gaps in the current state of knowledge • The major factor in this program 
is the need for rnethods which differentiate individual^ multi-individual y and team 
skills and training requiremeiits * Such techniques, which could be integrated with 
job/task analyses, would discriminate between isolated and interactive behaviors • 
Possible techniques to study for this purpose are variations of interaction analyses 
and other unobtrusive measures. Successful development of this methodology will 
provide an opportunity to est team performance standards criteria as part 

of training requirements analyses. Development of these techniques would then 
permit other research studies to be accomplished effectively. 

Issues identified in this review which merit research support include the 
following: 

• Team Feedback 

What changes in training methods or content result from feedback? 
Which method of providing team feedback is most useful (e.g., video 
recording versus post-exercise debriefing)? 

• Assessment Training 

Will training of team members to assess their own performance result in 
greater team proficiency than if no self-assessment training was provided? 
Will training to assess the performance of other team members affect 
proficiency? 

• Simulation Fidelity 

What degree of simulation fidelity is critical for effective team 
training? Can two-sided engagement simulation techniques be applied to 
the training of team skills? Although low fidelity simulations were 
found to be ineffective in established task situations, would they be 
more useful in emergent situations? 
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• Teamtomposltlon/Structure 

What are the effects of degree of team member familiarity on the 
acquisition of team skills? Will the length of time that a team functions 
together effect proficiency? 

• Skill Training Sequencing 

Is it more efficient to train individual skills first and then team 
skills^ or team skills first, or both concurrently? 

Combined studies are feasible which deal with more than one of the above 
research issues. For example, feedback or skill sequenc in ^issues can 
. be studied in environments which vary in their degree of simulation fidelity, 
or in task situations which are established or emergent - 

The studies that were suggested above point out the considerable research 
and development effort needed in the team training/evaluation area. With, further 
support, our state of knowledge in the area should expand considerably. 
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PREFACE 



This report contains a critical review of the literature relevant 
to team training. Current instructional and evaluative techniques within the 
Services were surveyed and described. State-of-art gaps were identified and 



research needs documented. 
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Chapter I: INTRODUCTION 

In terdependent^K coordinated team performance is a predominant character- 

istic of most operational activities within the Services • Training of teams in 

,/ 

operational utiits is the transition between initial training and combat. The 

ij 

influence of Wfective team behavior upon system performance underlines its 
consequences for training. This training is more difficult and assumed to be 
more costly to do than is individual training. It is usually performed in 
operational environments or in high fidelity simulations of them. For these 
reasons, there is a need for sound data to determine the relative effectiveness 
of alternate training concepts as a means for improving team training. 

BACKGROUND 

In order to pursue their mission of operational readiness, the Services 
conduct most (up to 90%) of their training in the operational commands. However, 
in the past, most training research has focused on individual training at 
Schools and at other institutional locations. 

MG Gorman, Deputy Chief of Staff for Training, TRADOC, recently stated [94] 
that there is a need for increased research emphasis on what is called "collec- 
tive training" (team or crew tactical training in a unit context). A recent 
Defense Science Board task force report [74] stated that in view of the amount 
of team training that goes on in the Services, either formally recognized as 
training or combined, with operations, the funds committed to R&D support for 
team training are relatively small. In fiscal year 1974, the Army Research^ 
Institute (ARI) spent less than $2 million on research in unit training. Although 
it was the largest such program of research in history, it amounted to only 11% 
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of the human resources research program in that year. Most of the money went 
for improved institutional training or exploration of other individual training 
issues . 

This review of the team training area was undertaken by HumRRO in order 
• to provide an information base that could be used by ARPA and the Services as a 
foundation to facilitate decisions regarding future research program support. 

PURPOSE 

The purpose of this review was to provide information to DARPA which would 
be useful for planning research and development programs in the area of team \ 
training. Such programs are needed to develop improved team training methods • 
and technologies and to provide new measurement techniques and procedures for 
evaluation • The following objectives were established to guide the preparation 
of this review: 

2. Describe existing instructional strategies and evaluation techniques 
relevant to team training. 

2. Identify state^of^ojpt gaps in team training and evaluation and 

suggest new team training strategies and evaluation techniques which 
warrant further study. 

APPROACH 

Review of Technical Literature 

An extensive search was made of the major documentation sources in order 
to amass an initial listing* of publications which were relevant to team training. 
The emphasis was on the applicability of the document to tactical training of 
teams. However, instructional strategies and evaluation techniques which had 
been used in other contexts were examined for their possible relevancy. The 
following major sources were searched: 
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!• Educational Resources Information Center (ERIC), These indices were 
searched for relevant titles over the preceding 10-year period. The descriptors 
used in this search' included: team train^ing, military training, industrial 
training, group dynamics, group effectiveness, group instruction, small group 
instruction, training techniques, 

2, Defense Documentation Center (DDC), The indices of the preceding 10- 
year period were searched using the following descriptors: team training, group 
dynamics, group problem solving, training crews, 

3, Psychological Abstracts, The descriptors used to search these indices 
were: group dynamics, training, training/indus trial, training/military, and 
group/small. Here too, a 10-year period was selected for the in-depth search, 

4, National Technical Information Service (NTIS), The general area of 
"military sciences" was canvassed for appropriate literature. Also, team 
training, group dynamics, group problem solving and group training were used as 
descriptors. 

5, HumRRO Library, This library specializes in publications which are 
relevant to military training and evaluation, A sizeable number of documents 
were thus obtained "in-house" for this review. 

Following the development of an initial list which contained many hundreds 
of titles, a brief review of each document's abstract was performed to insure 
that the review would focus only on those publications which were relevant to 
team training and evaluation. Relevant documents were then obtained and 
^ reviewed. 

Once the search in the initial sources was completed, secondary sources 
(i,e,. Journal of Applied Psychology^ Training in Business and Industry) were 
canvassed for relevant articles. In addition, "key" documents that were more 
than 10 years old were identified from the cross-references in documents 
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already reviewed. These "key" documents were also obtained and reviewed. All 
these items are listed in the References Section of this report. 

Sample of Current Military Team Training Techniques 

The most recent military doctrine regarding team training procedures was 
obtained and reviewed. Current studies underway within the Army Research 
Insi . ute and the other Services were examined. Military personnel were contacted 
in all the Services to identify team training approaches currently in operation. 
HumRRO personnel with experience in the team training area (e.g., Dr. Joseph 
Olmstead) were consulted for information about current military team training 
methods. Examples of current techniques were then chosen for inclusion in this 
report and are described in Chapter IV. 

DEFINITIONS 

As can be seen from the variety of sources chosen for the initial literature 
search described above, a clear conceptual delineation of the team training area 
had not been made at the time by HumRRO reviewers. Team training research was 
viewed as pare of tTie more general subject matter of small group research or 
group dynamics. This larger field had generated literally thousands of empirical 
studies and theoretical papers, most of which were only tangentially relevant 
to the sub-area of team training. An understanding was needed of what makes the 
team context unique. 

Before a state-of-art review on team training could be accomplished, one 
needed to define what constitut'?s" a "team.** In size, teams may e two or more 
individuals with no upper limlt^^^^^^^* Teams also may be characterized by 
virtually any degree of formal organizational structure and permanency [41]."'' 

"''This document by Hall and Rizzo provides an excellent description of how defini- 
tional problems have troubled the team training field and the various ways the 
research literature has dealt with these difficulties. 



Rather than attempting one more definition, the authors viewed the 

description givei;i by Glaser, Klaus and Egerman [33] as providing the most 

meaningful distinction of small groups and teams considered in this review. 

"Teams," have the following characteristics: 

"1. They are relatively rigid in structure, organization, and 
communication pattern. 

2, The task of each team member is well defined. 

3. The functioning of the team depends upon the coordinated parti-- 
cipation of all or several individuals. 

In contrast, 'small groups* differ in that they generally: 

1. Have an indefinite or loose structure, organization, and communica- 
tion pattern. 

2. Have assignments which are assumed in the course of group inter- 
action rather than designated beforehand. 

3. The group product can be a function of one or more of the group 
members involved depending upon the quality and quantity of their 
participation. " 

These definitions were used to guide the final selection of studies and papers 
for in-depth review. 

Instructional strategies and evaluation techniques which are relevant to 
"teams" should take into consideration the dimensions described above. In 
addition, teams are goal- or mission-oriented [41 ] and so^he specific context 
in which the team will operate must be considered before any training or 
evaluation technique can be applied. 
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The question of whether "team" or "multi-individual" behaviors are involved 
in a given context has not been adequately studied. As derived from the above 
discussion, team training can be defined as: 

# The training of two or more individuals who are associated 
together in work or activity. The team is relatively rigid 
in structure and communication pattern. It is goal- or 
mission-oriented with the task of each team member, well- 
defined. The functioning of the team depends upon the 
coordinated participation of all or several individuals. 
The focus of team training and feedback is on team skills 
(e.g., coordination) , activities , and products . 

On the oth(?r hand, multi-individual training can be defined as: 

# The training of two or more individuals who are associated in 
a group context. The trainees may or may not be part of a 
team. However,^ the focus of the- training and feedback is on 
individual skills, activities, and products. 

Methods are needed by which one can analyze given task situations and derive 
individual and team training requirements. 

In the chapters that follow, a distinction will be made on this dimension oi 
whether the study <)r technique reviewed is pertinent to multi-individual or 
team contexts. 
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Chapter II: REVIEW OF TEAM TRAINING RESEARCH 

Research in the area of team training has been going on for approximately 
20 years, albeit at a relatively low level of effort [21, 22, 37, 55, 62, 64, 70, 
71]. Glanzer and Glaser [35] stated that despite the importance of the area, 
relatively little for.nal knowledge existed concerning methods of describing and 
analyzing team performance. The complexity of team training problems was cited 
as contributing to that situation — "In the investigation of the areas of team 
as opposed to individual training and performance, problems appear of an entirely 
new order of magnitude*' [35]. 

In 1975 the Defense Science Board described several difficulties in perform- 
ing team training research which have prevented significant breakthroughs: 

This kind of R&D must be piggybacked on operations in the field, 
large numbers of R&D personnel are required, the opportunities for 
data collection during the exercise are marginal, inferential 
statistics and psychometrics were not designed for this order of 
complexity, there are limited opportunities for repeated trials, the 
ultimate test of team training is combat, which cannot be simulated. [74] 

The difficulties cited in doing team training research have prevented a 
thorough study of the problems and research needs identified during the inter- 
vening years. However, there has been some definite progress and certain 
research trends identified and described [36]. 

Research findings on some major team training problems and issues will 
be described in the eight sections that follow. The first section describes 
the ''-Stimulus-Response" and "Organismic" models which underlie different approaches 
to the study of team training [1]. Next (Section 2)^ the components of emergent 
versus established situations and their effects on team training research are 
described. A discussion of the relative importance and interactions of 
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individual training and team training (Section 3) is followed by descriptions 
of and problems involved in studying team skills (Section 4). In Section 5 
the degree of simulation fidelity in conjunction with its influence on effective 
training is examined. This is foj lov;^d by a description (Section 6) of the 
effects of feedback on team performances and techniques used to present knowledge 
of results (KOR) to team members. Section 7 examines team structure and 
composition and th =ir effects on team training. Finally, in Section 8 the 
benefits of a systems approach to the development of effective team training is 
discussed. 

1. TWO CONCEPTUAL APPROACHES TO TEAM TRAINING RESEARCH 

Alexander and Cooperband [1] have distinguished between two team training' 
models based on the situations in which team behavior takes place. The 
variables and environment selected for study by researchers depend upon the team 
model chosen. The "Stimulus-Response" model is applied to teams which operate 
primarily in established situations, whose tasks, and the activities required to 
perform these tasks, can be' almost completely specified, and the assignment of 
functions among team members and their equipment is relatively rigid. This 
viewpoint underlies attempts to apply certain principles arid techniques of 
individual learning (e.g*. , operant conditioning) to research on team training. 
Studies performed at the American Institutes for Research Team Training Laboratory 
are illustrative of the model [53]. For example, in one study reinforcement of 
team response was manipulated to measure differences in team member proficiency. 
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In the "Organismic" model [1] the team is considered to be a synthetic 
organism of which individuals are components. This model is oriented toward 
teams operating in environments which include a significant proportion of 
emergent: situations. Although there are defined task assignments, the individual 
has a considerable degree of discretion in how to perform the task under various 
contingencies. Consequently, proficient team performance can depend upon the 
development of appropriate team procedures for coping with the . environment by 
its members. Thus, adaptive innovations are required and decision-making and 
problem-solving skills are emphasi:2:ed [1]. An example of this is the work dorm 
by Chapman, ejt al [97]. Teams in a simulated air defense direction center were 
presented with sudden disruptive changes in their environment and were required 
to react to the new stimuli quickly in order to solve given problems precisely. 

2. ESTABLISHED VS. EMERGENT SITUATIONS 

The two conceptual approaches described in Section 1 take as their basis 
the situational context in ^jhich team behavior occurs. This context is 
actually a continuum of situations, the end points of which are described as 
established or emergent . Boguslaw and Porter [10] define these situations as 
follows: 

An established situation is one in which (1) all action- relevant 
environmental conditions are specifiable and predictable, (2) all 
action-relevant states of the system are specifiable and predictable, 
and (3) available research technology or records are adequate to 
provide statements about the probable consequences of alternative 
actions. An emergent situation is one in which (1) all action- 
relevant environmental conditions have not been specified, (2) the 
state of the system does not correspond to relied-upon predictions, 
(3) analytic solutions are not available, given the current state of 
analytic technology [10]. 

An established situation was the primary focus of a research program of 
the Team Trailing Laboratory of the American Institute for Research [54]. 
The advantages of such a situation for research are: 

19 



• it is relatively easy to abstract key characteristics of real- 
world team tasks and represent them in the laboratory; 

• experiments can be done relatively inexpensively; 

• studies can be conducted in reasonable time periods; and 

• monitoring of team activities can be feasibly accomplished. 

On the other hand, simplifying the representation of team functions can mask 
or omit possible important variables which influence behavior in the real world. 
By abstracting the situational context in the laboratory, there can be a loss 
of opportunity for trainees to react to breakdowns and problems which may arise 
in an operational setting. 

Providing skills to deal with emergent , unstructured situations was seen 
as a major goal of team training in studies by the Sys tem, Development Corporation 
(SDC) [1]. The development of coordinative skills was stressed alt\iough it was 
recognized that these are based upon a certain minimum attainment of individual 
skill. Team training devices and techniques were seen as requiring orientation 
toward the training of innovative behaviors and skills necessary to adapt to 
unforeseen problems [11, 23], 

An emergent situation permits a more realistic, less abstract, approach to 
the investigation of team training variables than does an established situation, 
li/hen a team was formed in the laboratory, its prior motivational characteristics 
were important determinants of performance. But, when teams were studied vrhile 
carrying out face valid activities, prior motivational states appeared to be 
less important [29]. What was important in this case was training in coordina- 
tion such that team members became fully aware of their responsibilities to 
compensate for the inabilities of others, or to overcome temporary problems 
when the situation called for it, 
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By viewing task situations as established or emergent, one can classify 
the training techniques and research studies discussed in this review as pertinent 
to either type of context. Similarly, the distinction between team and multi- 
individual training requirements discussed earlier comprises a second classifica- 
tory dimension. The resulting 2x2 matrix constitutes a classification scheme 
which will be used in the remaining sections of this chapter and in^ later chapters 
to categorize the research studies and training procedures surveyed."^ 

3. INDIVIDUAL VS. TEAM TRAINING 

The studies in this section were classified as shown in Table 1. Although 
the fccus of thi y literature review was on tOMm training, there were some studies 
surve^/'^d that dealt with individual skills training in a multi-individual context. 

TRAINING FOCUS 



TEAM 


MULTI-INDIVIDUAL 


Klaus & Glaser [51,52,54] 


Horrocks [43,44] 


Hall & Rizzo [41] 




Briggs & Johnston [11,14] 




Kanarick, et al [49] 


Kanarick, et al [49] 


Johnston [47] 




McRae [59] 




Williges [87] 




Alexander & Cooperband [1] 
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Table 1. Classification of studies reviewed in Chapter II, Section 3, 
"Individual vs. Team Training." 



The classification of studies throughout this document represent the judgments 
of the authors. No formal rating system was used. 
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One cotninon conclusion of the research on team training, regar.dless of 
whether it was based on an "Organismic" or "Stimulv's-Response" viewpoint, is 
that individual proficiency is the basis for an effective team [49]. Thus, 
in emergent situations, developing team awareness or abilities to deal with 
unexpected problems requires that each team member has attained the requisite 
job skills. Team training is likely to proceed most efficiently when the team 
members have thoroughly mastered their own specific assignments beforehand 
[54]. Horrocks, £t a]^ [43, 44] found that when team coordination was emphasized 
early in training, it interfered with acquisition of individual task competence. 

The question could then be asked if team training was of any benefit at 
all and, if so, to what degree and in what tRahrier. Is it possible to predict 
when individual training will be superior to team training? [14] Briggs and 
Johnston [11] concluded their review finding no direct evidence for the super- 
irority of team training over individual training. They interpreted this find- 
ing as indicating a need for team training devices to include provisions for either 
refreshing or augmenting individual skills. In other laboratory investigations 
(derived from established situations) which compared individual with team 
training, equivocal results were produced. Horrocks, et al [44] found that 
if a member of an intact team was replaced by another equally competent person, 
there was no detriment in team performance. This implied that there did not 
appear to be a generalized "team skill" which operated independently of specific 
task competencies [11]. Evidence supporting this finding came from other AIR 
laboratory studies [51, 52]. In these studies only the proficiency level of 
the members at the start of team training determined team performance. As these 
task situations were rputine and well-established, team performance was seen 
to be the sum total of individual performances [41]. 
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On the other hand, Johnston [47] studied two-person teams working in a 
simulated radar situation where the objective was to track and intercept 
incoming aircraft. The amount of coordination and communication in the training 
task was varied and demonstrated that team training was better than individual 
training when the criterion task required coordination between individuals. 
In such emergent situations, training in "team skills" (e.g.. coordination) 
was required and effective performance was regarded as something more than the 
sum of individual skills [59]. 

Verbal communication training was found not to be useful when the task 
did not require the skill [47]. In a study by Williges, et al [89] it was 
concluded that verbal communication played an insignificant role in team work 
and this role was not enhanced by verbal training. In this study, two-person 
teams were required to coordinate interception of two planes on a simulated 
radar screen. In one condition team members received only verbal communication 
concerning the performance of the other team member; in another condition, the 
team members were given verbal communication and visual access to the radar screen 
of the other member. It was found that verbal communication facilitated 
performance of the team only when there was no other more efficient communication 
channel available. In the conditions where verbal and visual communication both, 
were allowed, verbal communication had no effect on team performance. 

Alexander and Cooperband [1] proposed that team training should be 
effective when: the training stresses the acquisition of coordinative skills, 
and the tasks being trained are such that formal rules cannot be stated a;id 
procedures must be developed by the team in the process of task accomplishment. 
Kanarick, et al [49] proposed three phases for team training; 
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Initially, there is the need to train individuals in the procedural 
aspects of their jobs, doctrine, and the process approach to decision 
making. This training should be followed by a phase in which team 
members are instructed as a unit, learning the interactive and 
communicative requirements of team functioning. The final phase is 
devoted to tactical training where teams are taught to apply their 
procedural and interactive skills to certain situations requiring inno- 
vative and creative behaviors. 

4. TEAM SKILLS 

Studies in this Section are classified in Table 2, 



TRAINING FOCUS 

TEAM MULTI -INDIVIDUAL 



G 
W 


• ■ 










. M 






O H 












h- 






<t 


Sidorsky & Simoneau [80j 
Sidorsky & Houseman [81] 




ID 
h- 

V—* 


Hausser, et_ al [42] 


00 






H 
W 


Alexander & Cooperband [1] 










Federman & Siegel [27], 






Boguslaw & Porter [10] 
Kanarick, et al [49] 





Table 2. Classification of studies reviewed in Chapter II, Section 4 
"Team Skills." 



As can be seen, training in team skills is important for application to emergent 
situations [80,81]. What are these skills and how should they be trained? 

24 

22 



ERIC 



With respect to one such skill — cooperation — Alexander and Cooperband [1] have 
described it as follows: 

Learning to cooperate means learning the strengths and weaknesses of 
one another, learning when the others want help and when they do not 
want it, learning to pace one's activities to fit the needs of all, 
and learning to behave so that one's actions are not ambiguous. 

Nonetheless, one of the problems in the area of "team skills" is a lack of 
unambiguous, operational definitions of skills such as coordination, interaction, 
cooperation, etc. [27]. As there are conflicting views regarding these terms, 
so too are there problems in training and evaluation of these skills [91]. 

One of the most difficult skills to train, yet one of the most important 
in dealing with team functions, is skill in the analysis of one's own errors. 
Boguslaw and Porter [10] reported that it did not appear possible to specify 
how to train this skill. It was hypothesized that it would help to present 
trainees with numerous unusual situations which could not be handled by simple 
adherence to standard operating procedures. Problem-solving discussions (rather 
than didactic presentations) also could have helped. An effective team performer 
must learn when he/she is reaching the point of being overloaded and needs assistance. 
Studies are needed to examine the kinds of "self-evaluative" skills which can 
enhance team performance and to identify techniques which could be used to train 
these skills. 

The training of team awareness, a skill considered of importance to performance 
in emergent situations [49], can benefit from the application of a number of 
existing technologies. The split- or multiple-screen technique of television can 
be used to show team members their jobs in relation to other team members. 
Using this technique, a trainee could observe how a given action serves as a 
stimulus for some other team member. Such training provides the trainees with a 
better appreciation of their individual roles in the team and of the need for 
effective communication and interaction, 
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A study was conducted to investigate the application of the ^PLATO IV Computer 
Assisted Instruction (CAI) system to training interpersonal skills The 
content areas chosen included: feedback, communication, goal setting, problem 
solving, decision making, and reinforcement. Training had a positive effect on 
some of the skill performances of company commanders in leading recruit companies 
through a nine-week training period. However, in most of the behavioral measures 
of company commander performance, CAI training did not have a discernible effect. 
Further investigation of the usefulness of CAI as an approach to training team 
skills appears needed. 

5. SIMULATION FIDELITY 

Studies discussed in this Section are classified as shown in Table 3. 
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Briggs & Johnston [11-14] 




Bachrach [3] 

Alexander & Cooperband [1] 


Madden [60] 
Prophet & Caro [73] 



Table 3. Classification of studies reviewed in Chapter II, Section 5, 
''Simulation Fidelity." 



In training teams or studying their behavior in emergent situations, high 
fidelity simulations may be contemplated [60, 73]. High-f idelity-aitfTulation 
can be both costly and time consuming; space, equipment, and staffing constitute 
substantial investments in money and t^'me. The problems of measurement are 
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great because of the interactions of the many variables involved. Once a large- 
scale simulation exercise has begun, the experimenters have little control over 
the events which follow [3]. 

High-fidelity simulation of complex team tasks are exemplified by those 
which simulated air traffic control tasks and air defense operations which 
utilized teams ranging from 30-40 people. These tasks approached very closely 
the complexities of the operational task, and task fidelity was very high since in 
some cases an operational system itself was used in training [1]. 

Can low-fidelity simulations be used adequately for team training? A 
series of experiments by Briggs, eit al [11, 12, 13, 14] have led to the conclusion 
that team training did not transfer well to the operational environment when the 
training sinnilation was of low fidelity. However, these studies were all performed 
in contexts derived from established situations. No studies were found using low- 
fidelity simulations in emergent situations. Recent techniques used in the Army 
to simulate two-sided combat engagements have employed low-cost, high-fidelity 
procedures in emergent contexts. These techniques will be discussed further in 
later chapters of this report. 

6. FEEDBACK/KNOWLEDGE OF rSuLTS 

Studies on performance feedback discussed in this Section were classified 
as shovn in Table 4. Most of this research was performed in laboratory 
settings. 
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Table 4. Classification of studies reviewed in Chapter II, Section 6, 
"Feedback/Knowledge of Results. 

"Performance feedback is unquestionably the single most critical parameter 
in team or individual training," concluded Kanarick, et^ al [49] in a review of 
the research literature on feedback. The series of studies by AIR [25, 26, 33, 

51, 52, 54, 77] considered an important characteristic of teams to be the fact 

/ 

that regardless of how a correct team response was made, every member of the 
team was exposed to the same reinforcing event. This was true even when some 
team members made errors or were idle. These studies demonstrated that team 
reinforcement might also foster um-raiXted, inappropriate responses that could 
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create a significant decrement in team performance. The results of this program 
of research suggested several guidelines which should be employed for team 
^^training. 

... it appears essential that team practice result in clear 
and immediate reinforcement following each correct team response; 
practice in the absence of team reinforcement for criterion-level 
performance is more than likely to lead to a decrement in team 
proficiency [54]. 

An approach used to provide team members with feedback or Knowledge of 
Results is the post-exercise debriefing. Debriefing sessions following team train- 
ing sessions are an opportunity for the team to examine individual proficiencies 
and to explore alternative ways of organizing the task so as to develop more 
efficient and proficient team performance. However, the post-exercise discussion 
as feedback may have an undesirable characteristic—a large time gap between the 
response and the feedback [34]. The amount of elapsed time before feedback is 
given should be studied systematically to determine the effects of delayed 
feedback. 

In a study by Nebeker, et _al [65], the results showed that although 
individuals performed better with feedback than without, it did not matter 
whether the feedback was raw or percentile scores or whether it concerned the 
individual, his/her group, or both. The results underscored the importance of 
feedback as a general factor which augmented and sustained performance. Other 
studies of feedback j^g^demonstra ted that reinforcement conti|igencies affect 
team performance, ani^ve illustrated the effectiveness of feedback or Knowledge 

■ -1 

of Results to sustain performance [15, 53, 61, 90, 92]. 

Team members receive feedback information from intrinsic and/or j^trinsic 
sources. Extrinsic feedback is provided by a source external to the system, such 
as an instructor. Intrinsic feedback is received while the team is performing its 
tasks and is inherent in the task itself. The effectiveness of intrinsic feedback 
has been shown in a variety of studies [18, 19]. 
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Providing feedback in team training presents problems beyond those 

encountered in training individuals- 

These problems arise from three considerations, (1) There 
may be several criteria of effective team performance with no clear- 
cut tradeoffs among them; these criteria may be vague and difficult 
to state objectively and may change during system operations, (2) 
In order for a team to operate effectively, it is necessary for its 
members to develop and maintain individual skills as well as skill in 
working together; there is a possibility that these skills may 
require different feedback procedures which may mutually interfere, 
(3) When a complex system operates, there is usually a large volume 
of information available about the state of the environment, the 
state of the system, and the performance of system personnel; some 
of this information may be conducive and some inimical to effective 
learning [!]• 

A major source of difficulty in team training is the identification and correc- 
tion of individual errors. In many complex team tasks, there are often many 
possible correct procedures, making error identification difficult. These 
conditions give rise to the problem of monitoring or supervision of team 
responses to evaluate their adequacy. 

Although the studies cited above discuss the general effectiveness of 
feedback, more work needs to be done to systematically investigate the methods 
which might be used to deliver feedback information. Various techniques such 
as videorecording and post-training debriefings are now used to inform team 
members about their responses. These and other methods need co be studied 
regarding their effects on team performance, 

7, TEAM STRUCTURE AND COMPOSITION 

Studies discussed in this Section were classified as shown in Table 5, 
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Table 5. Classification of studies reviewed in Chapter II, Section 7, 
"Team Structure and Composition." 

Although not directly related to team training techniques, questions 
regarding the effects of group structure and composition upon training are 
important for understanding the effects of training upon team performance [6]. 
Two types of team structure were identified and studied in a series of laboratory 
experiments [11]. In a serial or vertical structure activities are sequential 
with input from one team member based on output of another. Parallel team 
structures (e.g., rowing team) do not have the same member interactions. 

Briggs and Johnston [13] suggested that parallel team structures are pre- 
ferable to serial structures. This was because team performance in the parallel 
structure was not dependent on the least skilled member. However, Klaus [54] 
reported that the parallel structure led to only a short-term gain and eventually 
to a decrement in team performance. 
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A variety of studies on team structure, workload allocation, physical 
arrangement, and member replacement yielded conflicting results [38, 39]. For 
example, Bolin, e^ al [9] found in studying task size and team performance that 
fully cooperative three-man teams performed better than individuals working alone 
on image interpretation. Moore [63] and George [30] studied the effects of task 
characteristics on team performance in emergent settings. Moore found that 
increasing the task load on teams did not affect team performance when all the 
information necessary for task completion was furnished to the team. George 
showed teams performed better when they were required to use a communication 
network which included all of the team members. j In a study conducted in an 

established task context to test the effectSffbf team arrangement on performance 

\^ 

of radar controller teams [48], investigators'^f ound that team communication inhibited 
team performance in some cases. In several studies occurring in established 
task situations, team member replacement was found to be relatively unimportant 
and dependent on the skill levels of the persons involved [24, 25, 26, 39]. Yet, 
personnel turnover was considered to be a major factor degrading team performance in 
an operational setting. For example, a tank commander reported that with 40% 
turnover in crews every 90 days, one cannot expect acceptable levels of team 
proficiency [74]. There is a need to systematically investigate the effects on 
team proficiency of the length of time a team performs together under established 
or emergent conditions. 

With such conflicting findings, the search for superior- team training 
methods has been slow. Task- or situation-specific characteristics may account 
for many of the conflicting findings obtained in team training research. A 
more systematic approach to the research area appears necessary. This suggestion 
is amplified in the sect:|^ below. 
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8. SYSTEMS APPROACH TO TEAM TRAINING 

The most common framework for research in the team training field has been 
to study team behavior in the same manner in which individual behavior was 
investigated. The team was viewed by the "Stimulus-Response" adherents as a 
**single response unit" [54]. The team product as a whole was the focus of 
research rather than its component parts. This made it feasible to investigate 
the effects of various conditions of feedback upon team proficiency. The 
fact that the team response might have been quite complex and result from the 
integration of separate responses of several team members was of interest only 
to investigators who followed the "Organismic" viewpoint. 

Both sets of researchers favor the application of a "systems approach" or 
instructional system development (ISD) model to the design and development of 
team training exercises, materials, methods, and devices. Iniplementing such an 
approach could yield information necessary for the identification of critical 
variables to be studied as part of an extensive program of team training research. 

Teams are created or defined to accomplish certain goals or missions [28]. 
The relationships among team members are meaningful only to the extent that they 
are involved in attainment of the goals. Thus, before any team training is 
undertaken, an in-depth analysis of the system should be performed [72]. The 
task characteristics need to be identified and training objectives derived [40, 46] 
The members of the team should be made aware of the team's goals and objectives. 
Exercises and instructional materials should be structured with respect to the 
objectives, and the training situation should be made as similar as possible to 
the operational criterion situation [8, 34]. Finally, the quality control 
components of the system should provide feedback data to trainees and trainers 
for the purpose of improving the training system and team proficiency. 
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The major difficulties to overcome in order to implement an ISD approach to 
team training are its expense, the need for qualified personnel, and obtaining 
the resources and authority to carry out the training research and development 
deemed necessary. 

SUMMARY 

Two approaches to the study of team training were described. These 
research trends were based upon what was termed the "Stimulus-Response (S-R)" 
and "Organismic" models. The former model led to laboratory investigations 
which addressed team training problems occurring in established task situations. 
Adherents of the Organismic model tended to study team training in more realistic 
surroundings, as they were more interested in the emergent phenomena present in 
the task situation. The studies performed by both group of investigators 
cited above suggested the following conclusions: 

• Team training is a necessary addition to individual training for 
tasks whicli require interaction and other "team skills." 

• Effective team training can only occur if the team members enter 

the training situation with the requisite individual skill competencies. 

• The team context is not the appropriate location for initial 
individual skill acquisition. 

• Performance feedback is critical to team as well as individual 
skill acquisition. 

The research findings discussed above which show that a particular team 
structure is "better," or that one type of reinforcement schedule is more 
effective than another, are quite fragile results. They can be largely obviated 
by such things as lack of fidelity in the training situation [16]. The nature 
of the training situation is a critical element to be concerned with for future 
team training research. Up to now the tasks used to study team training, with 
some notable exceptions, have been quite limited. 
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Recent: programs in the Army have demonstrated the possibility of developing 
high fidelity, yet low-cost simulations which may provide team training 
researchers with the controls they need to study "team skills" in realistic 
environments rather than in the laborato*ry [74], Other implications for research 
generated by the review above will be discussed further in Chapter V, Finally 

the research literature has shown that weaknesses exist in conceptu^^l^d^ef ini- 
tions of such terms as "team," "coordination," "fidelity," "interaction," etc. 
E.'forts are needed to provide a more precise taxonomic underpinning to this field 
to serve as a basis for more generalizeable and comparable/ research. 
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Chapter III: REVIEW OF EVALUATION TECHNIQUES FOR TEAM TRAINING 

The importance of methods for systematically evaluating team training 

effectiveness and the difficulty in developing such methods has-been widely 

acknowledged for a number of years [32, 33, 34, 67, 74]; Glanzer and Glaser in 

1955 [35] noted that there was an absence of satisfactory proficiency measures 

for team performance. In 1962, Glaser [31] remarked: . 

With respect to proficiency measurement, less work has been done 
in this area than has been carried out with individual performance, 
yet the importance of developing proficiency measures for multiman 
systems is being more frequently considered. 

By 1974, Ob'ermayer, et al [67] reported that an economically acceptable 
means of objectively measuring behavioral skills in the team training setting 
was an elusive goal. This conclusion was echoed by the Defense Science Board's 
1975 review [74] of research and development programs in military training. 
Their review stated that the team performance measurement problem, was a 
"fundamental stumbling block to progress" in improving team training. 

While a systematic widely applicable methodology for team training evalua- 
tion is not yet at hand, there appears to be a reasonable consensus as to what 
the methodology ought to accomplish [8, 17, 50, 67]. The literature reviewed 
suggested that any team performance measurement system must address the following 
areas : 

1. The definition of team performance objectives in terms of specified, 
observable outcomes to include criteria for acceptance and conditions of per- 
formance. 

2. The definition of a metric or range of values applicable to each 
specified observable event.- 
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3. The detection, measurement, and recording of the value of an observed 
event at each occurrence. 

4. An evaluation of the team as having attained or not attained the 
objective — based on discrepancies between outcome criteria and observed event 
values . 

5. The feedback of team performance data to the training environment. 
The remainder of this chapter will discuss current developments in team 

training evaluation in terms of the first four areas listed above. In terms 
of the two-dimensional categorization of training focus and task situation. 
Table 6 shows'^hat most of the studies reviewed from this chapter are related to 
the evaluation of team training in emergent task situations. 

The Definition of Team Performance Objectives 

Much lias been written on procedures for deriving training and performance 
objectives [2, 82]. The actual statements of objectives are the end products^ 
of. a series of steps in a systems approach to train^.ng development. Under this 
approach, the resulting objectives have three characteristics: they must describe 
behaviors which are to be demonstrated in the test situation; they should indicate 
the conditions under which these behaviors are to be demonstrated; and they 
should include criteria or standards of performance. The absence of performance 
criteria, usually stated in terms of accuracy, time, casualties taken, tolerance 
limits, etc., appear to be a seriously deficient aspect of team training. and 
performance evaluation. 

2 

The Army Training and Evaluation Program (ARTEP) , a body of training 
reference literature stressing a mission-oriented, performance-based approach 



^The feedback of team performance data was discussed in Chapter II. 
2 

See Chapter IV for a more detailed description of ARTEP. 
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Table 6. Classification of studies /procedures reviewed in Chapter III. 
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has been under development: since 1974. In 1974-75, the 9th Infantry Division 
and the 1st Cavalry Division trained and tested under prototype Army Training and 
Evaluation Programs (ARTEPs) as part of an extensive examinat;i^on of the feasibility 
of ARTEP implementation. One of the primary objectives of the effort was to 
"assess the effectiveness of the ARTEP as an instrument for evaluating unit 
proficiency" [93]. It was concluded that the standards of performance stated for 
Army Training and Evaluation Program missions required revision. Army Training and 
Evaluation Program standards were inaccurate in some instances and too general and 
vague in many others, requiring individual rater judgment to a greater extent 
than necessary. Army Training and Evaluation Program standards, while ostensibly 
stated in specific performance terms, contained many indefinite terms such as 
"on time,'' "excessive," "sufficient," "promptly," "proper," etc. Interpretation 
of such terms is bomv:-. to vary widely without some guidance. One major difficulty 
relates to the use of multiple standards and the lack of guidance as to the 
relative value of each component. Such heavy reliance upon judgments of individual 
evaluators can lead to diminished Army Training and Evaluation Program validity 
and reliability. 

Obermayer [67] described a highly sophisticated, semi-automated system for 
evaluating air crew performance. He stated that in some areas, such as air 
combat maneuvering, combat readiness determination is purely subjective. Chesler 
[17] asserted that standards for many combat situations — simulated or real — 
are often matters of widely differing opinion. Larson [57] described a possible 
way to overcome these differences. In his survey of team performance effective- 
ness measures - suggested the use of the Delphi technique — a method for systematically 
extracting expert opinion — to elicit subjective indicators of team performance 
from senior Marine Corps personnel. These performance indicators could then be 
used in field exercises by less experienced evaluators. 
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Definition of Observable Event Metrics 

The critical concern in proficiency evaluation is whether or not a team 
has gotten the job done. It is the objective of evaluation indices to measure 
data that indicate whether the defined mission has been successfully performed. 
Precise specifications of events to be assessed could be advanced with the 
implementation of the Instructional System Development (ISD) approach which 
subsumes the use of job/ task analysis procedures. However, much performance 
evaluation in training situations today is still highly subjective [17]. 

Upon examination, this problem inevitably boils down to a lack of criterion 
performance variables which are objective, recordable, discriminatory, and — most 
important — acceptable to a consensus of persons familiar with the tasks and 
skills of concern. This is the criterion dilemma, which is especially severe 
for team training. Acceptable indices of good and poor performance for military 
operations, are difficult to define [17]. Nonetheless, techniques for the 
systematic identification* of measurable, related events have been developed. 
[8, 17. 67] 

Adoption of such methods may well provide the kind of comprehensive, detailed 
information necessary to the development of valid performance criteria, but 
which at present is generally not available to training and evaluation* personnel. 
The Tactical Advanced Combat Direction Electronic Warfare (TAODEW) System appears 
well thought out and could readily serve as a model approach for other team 
performance evaluation efforts. 

The basic events in the TACDEW evaluation approach [17] are the "performance 
variable" and the "situational variable." 
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A performance variable is a measure of the accuracy or timeliness 
of a trainee response, A situational variable is a measure of some 
aspect of the embedding environment for the trainee's response. 
Performance variables derive from two types of data elements: actual 
trainee response, and the response considered correct. The performance 
variable is the quantitative difference between or qualitative compari- 
son of the two. For example, reported bearing (trainee response data 
element), for a target is 320 degrees; the true target bearing (correct 
response data element) is 300 degrees; the performance variable, error 
in reported bearing, is 20 degrees. This performance variable is 
a measure of accuracy of response [17]. 

Performance variables relating to timeliness of trainee response are 
derived from the actual latency of the trainee's response and a reference or 
ideal response latency, 

A set of performance (and situational) variables is then related to an event 
Once the event occurs, preliminary lists of performance variables are generated 
which identify any and all variables that show promise of serving as measures of 
training objective attainment. The developers of the Tactical Advanced Combat 
Electronic Warfare (TACDEW) system anticipate that their approach will help 
clarify the relationship among the various measurements and their relevance as 
criteria for making adequate judgments regarding team training outcomes. 

Using similar, but perhaps less clearly articulated methods, training 
evaluators in other Services are attempting to define objective-related events. 
For example, in Project REALTRAIb^/ au Army-supported approach which uses realisti 
two-sided, free-play exercises, a casualty assessment technique was developed for 
the basic infantry weapon, the M16A1 rifle. The development of the casualty 
assessment tectjifique for the M16A1 and the development of similar techniques for 
other infantry and armor weapons was considered a breakthrough necessary in 
simulating the tactical environment [79, 84], As part of the REALTRAIN procedures 
a sequential record of events is kept during each engagement at the controller's 



^See Chapter IV for a further discussion of REALTRAIN. 
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station. This record includes each casualty, the time at which it occurred, and 
the weapon used. 

Statistical analysis can help to reduce the number of possible variables. 
Some variables may correlate highly with others, and can be eliminated. Other 
variables may show very little spread in range of scores, so that they do not 
differentiate among trainees. Project NORM (Normative Operations Reporting 
Method), was an effort by the System Development Corporation to identify valid 
performance variables for SAGE [17] team evaluation, and to determine how 
situational variables affected team performance [76]. A basic concept of 
Project NORM was that situational variables were measures of exercise difficulty. 
An exploratory pilot study provided data for 41 variables — 24 performance and 
17 situational variables. Correlational, linear multi-variate regression, and 
factor analysis techniques were used to reduce the original number of 41 
variables to 13. [96] There were six situational variables (related to the task 
at hand) which were potential contributors to performance. In addition, seven 
performance variables (e.g., detection latency) were considered as relevant for 
evaluating system performance. Whereas the variables selected appear to be 
useful primarily to the study content, the analytic methodology employed to derive 
these variables is applicable to emergent task situations, in general. 

Biel [8] described another SAGE (Simulated Air-Ground Environment) study 
in which all measures used to evaluate team performance in a simulated environment 
were collected during a series of specially prepared simulated exercises. 
Correlations among team performance variables with measures of overall system 
performance in detection, identification, tracking , interceptor commitment, 
and interceptor guidance were computed and the resulting matrix factored. The 
results of this study recommended the deletion of those measures which did not 
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have a significant relationship to overall-all system performance. It also 
recommended that a small number of systematically obtained observations be used 
to make judgments about team performance. 

In Army tank crew training a large number of performance objectives have 
been developed for tank gunnery training and proficiency assessment. Project 
IDOC (Identification of Test Doctrine for Cost-Ef f ective Qualification of Tank 
Crews) is concerned with the identification of a reduced set of tasks on which to 
base the development of cost-effective performance measures that are applicable 
to the entire set of objectives. Accordingly, a joint HumRRO/American Institutes 
for Research effort was undertaken to determine the component team behaviors for 
each objective, and the commonality of behaviors across objectives . Plans are' 
currently being formulated_t_o analyze data obtained so far of generalities 
across tank gunnery tasks, psychometric problems, and practical constraings on 
performance assessment. 

Detection, Measurement, and Recording of Observable Events 

A problem of some concern is the unreliability and inaccuracy of trainers 
and observers in evaluating team performance in the field. Since there is no 
fixed doctrine of what\o^ observe and hoiv to evaluate it for various team acti- 
vities, the tendency is for inter-rater reliability to be quite low. 

Larson's 1971 review [57] of the team performance measurement literature 
concurs with this assessment. He stated that there was a shortage of objective, 
quantitative methods available for application in field exercise environments 
caused by an inherent difficulty in the measurement of complex human performance 
in a field environment. In REALTRAIN field exercises, detection and measurement 
of observable events (casualty assessment) is performed entirely by human 



^^J^^^n^^^ ^^^^ Research Institute staff. 
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controllers [76]. The light anti-tank weapon (LAW) and 90inni recoiless rifle (RCLR) 
are equipped with range-calibrated sighting plates which a controller looks 
through during a simulated engagement to determine hits. Casualties with the 
anti-tank weapon (TOW) are determined by a controller sighting through an 
accessory telescope mounted on the outside of the weapon. In a recent field tryout 
of the REALTRAIN method for combined arms tactical training, it was reported that 
the data collected were limited due to the methodological problems of quantitatively 
measuring team performance [79]. 

Some tesim training evaluation efforts attempted to circumvent the "unreliable" 
observer p'roblem by automating detection and measurement as much as possible. 
This is particularly true in areas where sophisticated electronic and computing 
technology are employed in mission fulfillment [4 , 8, 17, 67]. The Army 
is presently irves tigating the use of laser technology for this purpose. Project 
MILES (Multiple Integrated Laser Evaluation System) involves the integration 
of lasers with a variety of weapons types . "Firing" of a weapon emits a weak 
laser beam which, if aim is accurate, activates a recording device on the target. 
The MILES (Multiple Integrated Laser Evaluation System) is still experimental 
and has not been subjected to field trials, yet it has good potential for solving 
the problem of unreliable human measurement. Even though MILES was developed 
for use in multi-individual training settings, the concept could be applicable 
to the evaluation of crew-served weapons systems training. 

\ 

Where such technological solutions are not to be used, other techniques for 
minimizing rater effect are being explored. To support the development of 
systematic methods of criterion-referenced performance assessment, the Army 
Research Institute through project METTEST (Methodological Issues in Criterion 
Referenced Testing) is funding several investigations into the applicability 
of psychometric methods to the requirements of combat team performance measure- 
ment. One such effort addresses the question of which tasks, given limited time 
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and resources, should be assessed via high fidelity operational simulations, which 
may be measured "synthetically" (via paper and pencil instruments), and which 
may be assessed by inference from performance on related, more inclusive tasks. 

Evaluation of Objective Attainment 

Attempts to evaluate team performance during field performance tests have 
varied considerably. In some cases, it is assumed that if a given team completes 
an exercise successfully, the team received training in relevant vehaviors. 

The most common means of evaluating performance is overall subjective 
evaluation. Where detailed checklists are used, each individual team member 
receives a numerically weighted score. The team score, then, is arrived at 
through some weighted combination of individual scores — generally a simple sum. 

The ability of human evaluators to effectively judge- the success of teams 
in complex field exercises is generally considered unsatisfactory. In the Army 
Training and Evaluation Program (ARTEP) validation report [93] cited earlier, it 
was concluded that the ARTEP in its present form is not a standardized test 
instrument. There is no reason to expect that different teams would be evaluated 
under the same conditions when using Army Training and Evaluation Program 
guidance and standards. The standards are too subjective and evaluator performance 
is too erratic. There is no explanation o'f how to relate task performance to 
overall mission performance or how to adjust standards to account for varying 
test conditions [93]. 

The Modern Army Selected System Test Evaluation and Review (MASSTER) attempts 
to evaluate weapons systems training by employing a combination of checklists, 
associated logic trees, and expert judges in the field environment. The checklist 
logic tree evaluation uses field observers to check off the team's performance 
in specified situations. The team passes or fails progressively larger tests 
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as a result of their performance on previous examinations. MASSTER also assigns 
officers tc subjecti^-ly evaluate large teams during an exercise [57]. 

Currently experienced difficulties in making reliable and accurate judgments 
about team performance is in large measure due to the multiplicity of potentially 
influential factors. Chesler [17] stated: 

... single performance measures are inadequate for making overall 
evaluations of system, effectiveness, presumably because multiple 
factors are involved in the determination of mission success or failure. 
Combining measur.-.s into overall indices, has, so far, not seemed to help 
much, orobably because the relationship between them is not clearly 
understood. Thus, lumping them together does not necessarily improve 
the quality of system evaluation. 

Recognition of this complexity has led to a number of investigations of 
analytic or modeling approaches to the evaluation problem. A study within the 
Army Research Institute's METTEST (Methodological Issues in Criterion Referenced 
Testing) project is concerned with the applicability of traditional measurement 
models to the need for team performance indices. Measures are needed that give 
valid representation to the many team-related and environmental factors which 
influence mission attainment. Preliminary work has suggested that the pre- 
dominantly linear additive models of psychometrics are not applicable in this 
context—that non-linear (e.g., exponential) and "multiple-cutof f "..jnodels may 
be more appropriate. In "multiple-cutoff-models," team activities are 
structured in a manner related to the sequence in which they would occur in the 
field, and each is evaluated on a GO/NO-GO basis. Within this approach "compen- 
satory" and "non-compensatory" activities can be identified. It is non- 
compensatory if one particular component will abort the attainment of the mission 
if it is not performed properly. If some other component can successfully pick 
up the slack left by the failed component, then it is compensatory. 
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The UPAM model (Unit Performance Assessment Model) developed at the Array 
Research Institute employs casualty data as a measure of both the achievements 
of a team (casualties inflicted on an opposing force) and the cost to the team 
(casualties sustained) during its efforts to fulfill a mission. Measurement 
of a team's accomplishments is obtained by inputting to the model data from 
REALTRAIN field exercises and deriving a single value reflecting both cost and 
achievement. A criterion value for assessing the empirical result is obtained by 
having the unit commander provide estimates of these values that in his judgment 
are indicative of a successfully accomplished mission. The difference between 
empirical output and the commander's "expected" output (on a scale from -1 to +1) 
is viewed as an index of the team's success in attaining its goal. /^^UPAM model 
has been applied to both infantry and armor units. However, in its present state 
of development, the model does not provide evaluative information concerning team 
activities which contribute to or detract from mission accomplishment. 

The most detailed and sophisticated analytic model found in this survey 
is the CARMONETTE ground close combat cbmputer simulation [20]. Its parameters 
include detailed descriptions of team size, team type, firepower, vulnerability, 
mobility and sensing capabilities. Data on terrain ranges from elevation and 
height of vegetation to traf f icability. Additionally, a set of "orders" can 
be transmitted to each team of opposing forces (i.e., a mission). Further, each 
team can acquire information and relay it to equivalent teams or to its command 
HQ. While the output from CARMONETTE provides only descriptive data, it could, 
if provided with field data, provide valuable clues concerning the effect of 
performance and situational variables on mission outcomes. 
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Sumna ry 

There is still a major unsolved problem in the development of evaluation 
methods and tools for team training—namely, there is a need for reliable, 
objective-based, team performance measures. The literature reviewed indicated 
the difficulties of doing research in this area in spite of its acknowledged 
importance to the improvement of team training. The current support within the 
Army Research Institute for R&D in this area demonstrates their realization of it 
criticality. Most of this research is still in progress. Findings gleaned from 
the studies surveyed suggested the following: 

• A systems approach to team training development should 
alleviate some of the ^.vroblems associated with the establishment 
of criteria/standards. 

• Much performance evaluation in team training today is highly 
subjective and unreliable. 

• Lacking an adequate team performance measurement system, alternative 
team training programs and team proficiency cannot be properly 
evaluated. 

• Automating the monitoring and/or data recording process is needed 
in order to solve the problem of unreliable human observers. 

Recent innovations in team training methodology discussed in Chapter II 

promise significant improvements in operational performance. A strong emphasis 

on R&D in the area of team performance measurement is required in order that this 

improvement be detected and operationally realized. 
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Chapter IV: CURRENT MILITARY TEAM TRAINING APPROACHES 

Examples of team training techniques currently in use in the Services are 
presented in this section. It is not intended to be an exhaustive presentation 
of such training methods. Rather, the examples were chosen to represent the 
variety of methods now employed to conduct team training. The training devices and 
systems discussed in this chapter are categorized in Table 7. They were all 
classified as pertaining to emergent task situations because they stress the 
realistic requirements of operational environments. 

ARMY TEAM TRAINING 

Array team training is predominantly a unit training function with no formal 
institutional training requirements. The great majority of personnel who 
complete Military Occupational Specialty (MOS) training at Army schools and 
training centers are assigned to units as replacements. They receive team training 
in the units in the form of individual on-the-job training, augmentation, cross- 
training, and collective (team) training. Most training is performed on opera- 
tional equipment with the objectives of maximizing proficiency or readiness [41, 88]. 

In the Army, Advanced Flight Training is considered team training. Graduates 
of Undergraduate Pilot Training receive supplementary training in the specific 
aircraft they will fly on operational missions. The supplementary training is 
provided by the operational unit. The new pilot is integrated into the opera- 
tional unit where he undergoes transitional training as part of normal unit 
training [74]. 
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Much of the team training conducted in the Army is based on information 
contained in Army Training and Evaluation Program (ARTEP) publications. That 
is, ARTEPs "drive** the Army^s team training system. ARTEPs are developed to 
assist training managers and trainers in the preparation, conduct and evaluation 
of team training.^ They consist of a series of training and evaluation outlines 



TRAINING SITUATION 
TEAM MULTI-INDIVIDUAL 







CATTS (Combined Arms Tactical 
Training Simulator) [74] 

WST (Device 2F87, Weapons System 
Trainer) [41] 

Device 14A2, Surface Ship Anti- 
submarine Warf are(ASW) , Early 
Attack Weapons Systems Team 
Trainer [41] 

Device 14A6, ASW Coordinated Tactics 
Trainer [41] 

Device 21A37/4, Submarine Fleet 
Ballistic Missile (FBM) Training 
Facility [41] 

TACDEW (Tactical Advanced Combat 
Direction and Electronic Warfare 
System) [41 ,83] 

TESE (Tactical Exercise Simulator 
and Evaluator) [75] 

FIST (Functional Integrated Systems 
Trainer) [74] 

C-5 MissionFlight Simulator 

C-141 Flight Simulator 

C-130 Flight Simulator 

WST (B-52 Weapons Systems Trainer) 


SCOPES (Squad Combat Operations j 
Exercise, Simulation) [85,87] 

REALTRAIN [84] 

MILES (Multiple Integrated Laser 
Engagement System) [84] 

TWAES (Tactical Warfare Analysis 
and EvaluationSystem) [74] 

1 



Table 7. Classification of Training Contexts 



See Chapter III for a discussion of ARTEP as an evaluation mechanism. 
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which specify for a particular element of a battalion/task force, or separate 
coippany, the followirg information: 

a. The unit (i.e., crew, squad, platoon, company/team or batallion/ 
task force) for which the outline is applicable. 

b. The mission to be performed* 

c. The general conditions (situation) under which the mission will 
be performed, 

d. The primary training and evaluation standards upon which the unit 
will be evaluated as either satisfactory or unsatisfactory 

(Go /No Go), 

e. The collective training objectives applicable in the performance 
of the mission; guidance for estimating support requirements 
necessary to conduct formal evaluations, and tips to trainers and 
evaluators [85], 

Inasmuch as the Army Training and Evaluation Programs (ARTEPs) outline the 
basic missions that teams should be able to perform to be combat ready, they are a 
valuable source of information in preparing the scenarios for training strategies/ 
exercises such as SCOPES (Squad Combat Operations Exercise, Simulation), MILES 
(Multiple Integrated Laser Engagement System), REALTRAIN and CATTS (Combined Arms 
Tactical Training Simulator) which are discussed below, 

!• SCOPES (Squad Combat Operations Exercise, Simulation) 

SCOPES was originally designed to teach movement techniques to rifle teams 
and squads. It has been successfully used to train and/or evaluate soldiers 
or small teams on a wide range of battlefield skills, tasks, or missions. 

Briefly, SCOPES works in the following way. Trainers first develop a 
tactical scenario based on specified training objectives. Six-power telescopes 
mounted on each rifleman^s M16A1 rifle are used to identify numbers worn on the 
helmets of all participants in opposing forces, UTien a number of an "enemy" 
sold^.er is identified, the soldier making the. identification fires a blank^round 
and calls out the number he sees. Controllers in the vicinity in turn pass the"^^ . 
number over a radio net to controllers operating with the opposing force. If 
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the number is correct, the controller immediately *'kills" the man wearing the 
number seen. 

In this technique, "battles" may be conducted with real outcomes. Depending 
on the tactics selected and the individual techniques used, it is possible to 
objectively determine "winners" and "losers." 

After each engagement, troops on both sides are brought together for an 
after-action review to discuss error<: n^ade and lessons learned. In this manner, 
the correct techniques to avoid being "killed" can be brought out, learned, and 
practiced in subsequent engagements [85, 87]. 

2. REALTRAIN 

The REALTRAIN program permits the conduct of larger scale exercises using 
opposing forces than does SCOPES (Squad Combat Operations Exercise, Simulation) . 
To simulate the effects of other weapons systems, optical devices including 
telescopes and plastic sighting plates are mounted or used with light anti-tank 
weapons (LAWs); 106mm RCLR (recoiless rifles); anti-tank weapons (TOWs); tank 
man guns; DRAGON (t- .^i,.T.; cinti-tank weapon); and 90mm RCLR. These devices are 
aligned with the weapon's sight picture thereby allowing controllers to see the 
same sight picture as the gunners. This permits them to verify a gunner's aim 
during a target engagement. Gunners "shoot" at targets by announcing the identi- 
fication numbers worn by the soldiers aligned in their sights. Each optical device 
is correlated with the maximum effective range of the particular weapons system 
with which it is used [84]."^" 

Field tests conducted using SCOPES and REALTRAIN exercises have proven that 
engagement simulation techniques work for training ;nulti-individual groups. Both 
soldiers and units reach higher levels of tactical proficiency more rapidly, and 
soldiers are motivated to train because the tactical exercises present a real 
challenge. 



"^A more complete description of REALTRAIN can be found in Army Training Circular 
71-5 [84]. 
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3. MILES (Multiple Integrated Laser Engagement System) 

MILES a group of experimental training devices which uses an eye-safe 
laser beam to simulate the effects of weapons expected on the modern battlefield. 
The system will permit conduct of day and night tactical exercises and will 
improve the integration of gunnery techniques into tactical training. Some of 
these subsystems are: 

• Vehicle Engagement System (VES) . The VES will be used by armor units 
and anti-armor weapons such as TOW (tube-launched optically tracked 
wire guided missile), DRAGON (ir-dium anti-tank weapon), and the 106mm 
recoiless rifle. 

• Target Engagement Simulator (TES). The TES will be used by Infantry 
troops to provide engagement realism. It consists of a laser trans- 
mitter and a hit indication device which signals *'kills" and "near 
misses." 

• Machine Gun Laser (MGL) . The MGL simulates machine guns both unmounted 
and mounted on tankS". 

MILES will permit conduct of tactical exercises involving battalion and task 
force size teams [84]. It is scheduled for testing in 1978. 

The MILES procedure (as well as SCOPES—'Squad Combat Operations Exercise, 
Simulation—and REALTRAIN) focus on individual skills training in emergent task 
situations. The feasibility of using these techniques for training team skills 
needs to be determined. 

4. CATTS (Combined Arms Tactical Training Simulator) 

The purpose of the CATTS system which is still under development is to 

provide Battalion Commanders and their staffs with simulated combat situations 

operating from a ground command post. 

The CATTS system has several features desired for training. ' It allows 

realistic, real-time ground combat simulation; realistic mockups of command post 

vehicles; actual field radios for communication; on-the-spot command decisions 

and critiques; extensive automation to assist controllers; and cost-effective 

software and hardware. ^3 
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The simulator utilizes a Xerox Sigma 9 computer to meet the following 
training cojectives: 

a. Identify the relationships that exist among various elements and 
with the enemy. 

b. Identify alternative courses of action, appropriate formation 
maneuvers and fire support applications. 

c. Communicate decisions to subordinates using tactical orders , 
so that decisions can be effectively interpreted. 

d. Manipulate and monitor the variety of tactical radio networks 
available at the battalion level. [74] 

NAVY TEAM TRAINING 

As in all of the Services, team training in the Navy consists of training 
which is provided to organized teams for the performance of a specifdj^ mission. 
The Navy discusses team training in terms of five categories: Pre-team 
Indoctrination Training; Subsystem Team Training; System Sub-team Training; 
System Level Operational Training; and Multi-unit System Operational Training [74], 

Pre-team Indoctrination Training is conducted in a team context with emphasis 
on increasing the skill levels of the individuals who will later be assigned to 
operational teams. 

In Subsystem Team Training teams vary in composition under different 
conditions of shipboard readiness. The teams are divided into three departments: 
Combat System Teams (e.g., Search Radar Detection and Tracking); Unit Operations 
Teams (e.g.. Navigation); and Engineering Systems (e.g., Firefighting/Damage 
Control) . 

System Sub-team Training involves the training of two or more sub-system 
teams and generally an entire ship. 

System Level Operational Training is accomplished by at-sea- training 
because existing pierside facilities do not have the capability of handling an 
entire crew of a ship in all facets of an operation. This type of training is best 
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described as a battle problem which is conducted by underway training units and 
fleet training groups . 

Multi-unit System Operational Training is in-port exercises utilizing 
shore-based trainers for the purpose of training crews prior to getting underway 
for the exercise operations area. Major in-port exercises utilize the Tactical 
Advanced Combat Direction and Electronic Warfare (TACDEW) Trainer [74]. 

The following uajor team training devices used in the Navy are described 
below: Device 2F87 (Weapons Systems Trainer), Device 14A2 (Surface Ship Anti- 
submarine Warfare Early Attack Weapons Systems Trainer), Device 14A6 (Anti- 
submarine Warfare Coordinated Tactics Trainer), Device 21A37/4 (Submarine Fleet 
Ballistic Missile Training Facility), and TACDEW [41]. 

1. Device 2F87, Weapons Systems Trainer (WST) 

The WST duplicates the interior arrangement and appearance of the P-3C 
aircraft. Five trainee stations simulate corresponding stations in the aircraft. 
The' purpose of WST training is to teach team coordination. Students are organized 
into teams according to positions for which they are being trained. The trainees 
"fly" simulated anti-submarine warfare (ASW) missions. These missions are graduated 
in difficulty from very simpJe scenarios early in training to more complex 
exercises -toward course completion. 

2. Device 14A2, Surface Ship ASW (Anti -Submarine Warfare) Early Attack Weapons 
System Trainer 

Device 14A2 is used to train surface sliip teams in the proper utilization 
of operational ASW systems. Training emphasizes the procedural, tactical decision 
making, and team coordination activities in operating and empJoying ASW weapons 
systems. The device provides for indoctrination of personnel in ASW procedures 
and evaluation of tactical situations. The trainer is al-so used in developing 
^nd planning advanced Naval undersea defense tactics. 
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The trainer occupies over 3000 square feet of floor space and duplicates 
the physical configuration of major operational compartments and equipments of 
surface ship Anti-Submarine Warfare (ASW) attack weapons. It simulates their 
functional operation and responses such as target detection, fire control solution, 
and weapon launching and tracking. 

3. Device 14A5, ASW Coordinated Tactics Trainer 

Device 14A6 is designed to train decision-making personnel in the tasks 
they must perform when engaged in coordinated ASW tactics. Simultaneous operation 
of 48 vehicles of various types and a multiplicity of sensors can be simulated. 
Communications facilities simulate the various radio channels employed opera- 
tionally to coordinate all phases of an ASW mission from search through attack. 
Device 14A6 provides a synthetic environmer\L within whfch ASW personnel can 
practice collecting and evaluating ASW inf oririu t: ion , ■ .kinj:.> decisions, and imple- 
menting the decisions based on this int'cnr.jMon. The dovice is not intended to 
train equipment operators; therefore, simulated oquipmrnt does not resemble fleet 
equipment. Functional characteristics of ihe simulated equipment are similar 
to fleet equipment. 

Virtually any exercise at sea which requires ciommunication, coordination, 
maneuvering, and decision making may be practiced in the- 14A6 trainer prior to 
going to sea. 

4. Device 21A37/4, Submarine Fleet [ballistic Missile (FBM) Training Facility 
Device 21A37/4 provides training in offensive /and defensive tactics for 

nuclear attack center teams. Surface or subsurface maneuvers may be accomplished, 
and training may be given independently or in coordination with other units. 
Instruction of senior command and staff officers In direction and coordination 
of submarine task groups with surface support units may also be given. 
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A central digital computer provides problem generation, position and motion 
data generation. Up to 41 different vehicles can be included in training problems, 
A projection system in the attack centers permits both in-progress monitoring and 
post-fire analysis of training problems. Attack centers can be operated 
i.fidependently or operation can be coordinated to provide submarine versus 
submarine training. Fifteen different classifications of targets are currently 
available, 12 at any one time, 

5. Tactical Advanced Combat Direction and Electronic Warfare System (TACDEW)^ 
TACDEW is a highly sophisticated computer-based Simula tionfacility having 

three primary missions: individual and team training, tactics evaluation, and 
testing of operational computer programs for Navy Tactical Data System (NTDS) 
ships. Training is conducted in 23 Combat Information Center (CIC) mockups 
typical of the ships on which trainees serve [41], Team and multi-team training 
are accomplished. The purpose of TACDEW training is not so much to establish 
team interactive skills as it is to maintain or enhance these skills in simu- 
lated mission contexts. Training typically consists of exercising a given 
team, or some combination of teams, in a series of scenarios of graded difficulty. 
The scenarios arc desigaed to modtM tactical situations which might be 
encountered in an operational environment. 
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^'A more detailed description of the TACDEW svstem is presented in rhe TACDEV 
Information Pamphlet [83]. 
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MARINE TEAM TRAINING 

The Marine Corps has recently initiated two tactical exercise projects 
whose objectives are to develop and integrate computer-driven systems into 
ongoing field and school training programs. The computer-driven simulation 
which is the heart of these training systems, can handle a greater number of 
factors with greater precision and finer resolution than was previously possible 
in any manually conducted model [75] • 

The two team training systems associated with these projects are briefly 
discussed below: TESE (Tactical Exercise Simulator and Evaluator); TAWES 
(Tactical Warfare Analysis and Evaluation System), 

!• TESE (Tactical Exercise Simulator and Evaluator) 

TESE is used to train Marine Corps officers in combat decision making. 
The project seeks to define procedures for war gaming. The goal is to get both 
computer-based individual and team measures during amphibious warfare exercises, 
and to increase the number of trainees who can be processed- 

2. TWAES (Tactical Warfare Analysis and Evaluation System) 

TWAES is a computer-arrsisted system to control tactical field training 
exercises. The system offers potential improvements in maneuver control and 
simulation of indirect fire [74]. The current TWAES research effort is studying 
the exact role that the post-exercise session will play in the total" tactical 
exercise. The question being addressed is whether this feedback session will be 
a training vehicle to further extend the game's learning effectiveness or whether 
it will be merely a post-exercise debriefing during which administrative informa- 
tion about game procedures is passed on [75], 
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AIR FORCE TEAM TRAIN iwn 

In Che Air Force, Clu're is vory little formal team training being conducted. 
All such formal courses surveyed were used in training strategic missile teams. 
Students are provided training in the responsibilities and necessary technical 
procedures of tho appropriate weapons system, but their proficiency upon gradua- 
tion from team training courses is less than that required for combat-readiness. 
Thus, additional upgrade training is required at the unit/base prior to actual 
job performance [74-]. 

SiiTiilarly, there are few team training devices. One device uncovered in this 
survey is the Functional. Integrated Systems Trainer (FTST) . The purpose of FIST 
is to provide a betf.^r inbans for training four members of the fire control team 
on the AC-rjOi: Gunship. The second objective of FIST is to refine and promote the 
use of the technology for developing low-cost, interlinked, functional, part-task 
trainers [74]. 

Similarly, most team training devices deal with the training of aircraft 
flight crew members. Representative training devices are available for the B52, 
C5, C130 and Ci41 aircraft as described below. 

1. E52 (G & H models) Weapons System Trainer (WST) 

The i;ST is presently under development. It will permit the integration 
into a single team training device four individual devices presently being used. 
The four devices are: (a) The Mission Trainer used for training pilots and co- 
pilots; Cb) Navigator Trainer used for training radar navigators and navigators; 
(c) Electronic Warfare Officer Trainer (EWO) ; and (d) Gunnery Trainer. The WST 
when developed will permit simultaneous team training of the entire six-man 
hjZ crev;.-^ 



■^Personal communication with COL Roy L. Ripley, LTC Ualtor D. Black III, Instruc- 
tional Systems Branch, Audiovisual and Instructional Systems Division, US^ Air Force, 
Washington, DC. 57 59 



2. C5 Mission Flight Simulator 

This team training device consists of three training stations which permit 
integrated team training pilots/co-pilots, navigator and flight engineer."'" 

3. C130 Flight Simulator and C141 Flight Simulator 

Both of these training devices are similar to the C5 Mission Flight Simulator 
except that training is not provided for the navigator station. However, the 

C130 Flight Simulator is scheduled to add the navigator station as part of the 

2 

device in 1981." 

4. Functional Integrated Systems Trainer (FIST) 

The purpose of FIST is to provide a better means for training four nembers 
of the fire control team on the AC130E Gunship. The .se^.cond objective of FIST is 
to refine and promote the use of the technology for developing low-cost, inter- 
linked, functional, part task, trainers [74]. 

SUMMARY 

The emphasis in this survey has been on forrrial team training. M^ch informal 
team on-the-job~training occurs in the daily operations of the Services. It if 
extremely difficult to delineate, in such activities, the training mode from the 
operational. This is especially true in areas such flight training, air dofen^a 
etc . 



Personal communication with LTC C.R. Philbrick, MAC/bOTF, Scott AF3, IL. 
^Ibid. 
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However, various methods of team training are currently unden>^ay in the 
Services. These include the following: 

• Large-scale opposing forces tactical exercises 

• Devices which simulate effects of weapons 

• Computer simulations of tactical decision-making problems 

• Simulations of mission-specific hardware and software (e.g., anti- 
submarine warfare vehicles. 

The above methods are tailored for both initial team training and maintenance 

of already acquired team skills. They vary in cost, simulation fidelity, and 

generalizeability . Although these devices are now being used, there has been 

relatively little programmatic R&D performed upon them considering the cost of 

such systems. 
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Chapter V: RESEARCH IMPLICATIONS 

A number of implications for research in team training/evaluation are 
presented in this chapter. These implications resulted from a comparison 
of current military practice with the findings of the literature review. 
In addition, HumRRO staff relied on their experience in the training re- 
search area to identify the most critical areas for support. The specific 
questions addressed in this report were the following: 

(2) Wiiat state-of-czrt gaps are there in team training strategies and 
e va I ua tion teohniauea ? 

» 

(2) h^Mt new team training and evaluation strategies appear to hold 
prorm'se for applications to the DoD environment and warrant 
further study? 

The literature reviewed for this report support the contention of t.he 
Defense Science Board [74] that there is a great need for R&D in team 
training, considering the magnitude of such training in the Services. 
One major problem to overcome is the tremendous diversity of terms, con- 
cepts and operations which are used. A "team'' is not a simple fixed unit, 
it is arbitrary, has unclear boundaries, and an unstable structure and com- 
position. A team should be defined on the basis of its criterion situation, 
not on the basis of administrative convenience. [34] Describing team train- 
ing in the Services, the Defense Science Board [74] resorted to an awkward 
abbreviation, CGTU, for Crew, Group, Team, and Unit to accommodate users 
of each of these terms. There is a need for more standardization and sharing 
of terms. Efforts should be supported to develop a reasonable taxonomy 
upon which to build research programs in team training. 
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A classification scheme was introduced in Chapter 3 which was used to 
categorize the studies and techniques discussed in this review along two 
dimensions. The training focus can be either team or multi-individual — 
depending upon the skills to be trained. The task situation can be classified 
as established or emergent. 

Methods are needed for differentiating individual, multi-individual, and 
team skills. Techniques need to be developed which can be used to identify 
individual and team skill training requirements as part of job/ task analyses. 
Such techniques would discriminate between isolated and interactive behaviors 
and will provide the opportunity for establishing criteria/standards of team 
performance as part of training requirements analyses. Variations and/or 
combinations of observational interaction analyses and other unobtrusive 
measures which have been used in non-classroom settings should be studied 
for their possible application. A recent compendium [95] of such methods 
provides detailed information about techniques which are of potential value 
such as: 

• Interaction Process Analysis — Robert F. Bales 

• Sequential Analysis of Verbal Interaction (SAVI) — Anita Simon ' 

Yvonne Agazarian 

© CERLI Verbal Behavior Classification (CVC) — Cooperative Educational 

Research Laboratory, Inc. 

The development of such analytic techniques is a critical first step in 

any research program which seeks to use realistic team performance standards 

as effectiveness criteria. 

There is a considerable need for measures of team performance to apply 
both during and following training. As described in Chapter III this 
problem has resisted solution for the past 20 years. The difficulties of 
observing team member interactions, the great costs in money and time to 
conduct studies in this area, and the unclear relationships between task or 
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training variables and team performance, have all contributed to the un- 
successful search for measures of team proficiency. The production of 
standardizable , relatively invariant test conditions for evaluating dynamic 
and interactive team behavior remains one of the major problems in team 
evaluation. 

It is only in recent years that considerable research support has been 
provided to arrive at measures of team performance. Critical problems to over- 
come are: 

• The criterion problem — Without agreed-upon criteria or standards of 
team performance, evaluation of training or team proficiency cannot 
occur. 

• The measurement problem — Stated succinctly, what to measure, where, 
when, and how, are still all unanswered questions in the evaluation 
of team performance. 

Considering the current lack of information in this area, the potential 
heuristic value of the analytic or modeling approach is worthy of continued 
support. Of course, the usefulness of any model depends on its ability to 
generate information and ideas which lead to the development of practical 
and worthwhile evaluation tools and techniques. 

For example, the CARMONETTE simulation outputs predictions of team per- 
formance based on hypothetical values for a number of performance and situa- 
tional variables pertinent to the team and to an opposing force (e.g., size/ 
vulnerability, firepower, mobility, and/terrain features). Empirical values 
gathered from field exercise data for the variables could then be input to 
the simulation and outcomes compared. Continued systematic iterations of the 
model would suggest the specific variables that made a difference. If validated 
in further field exercises, these variables could guide the refinement of test 
situations and the development of better measurement techniques. 
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It is probably true that teams operate in situations which have a 
combination of both emergent and established characteristics. 

This underlies the importance of performing a thorough analysis of the 
criterion task situation prior to developing team training programs. An 
application of the systems approach to the development of team training 
should permit identification of the interactions, communication, coordination, 
decision making and other activities required in the task performance of 
each team member. The information gathered with the appropriate analytical 
tools should include standards and relevant conditions of performance. This 
would provide a major step toward obtaining clearly stated, objective criteria 
and procedures for evaluating team performance. 

One consistent finding of this literature review was that individual 
proficiency has been shown to be a major determiner of team performance. This 
had led investigators such as Hall and Rizzo [41] to conclude that more emphasis 
should be placed upon individual rather than team training. The authors of 
this review disagree. Although certain laboratory studies showed that when the 
tasks required only individual skills, team training was superfluous, other 
investigations, primarily in more realistic, "emergent" situations showed 
the importance of team training, \-men "team skills" (e.g., interaction), even 
though poorly defined, were important in the task situation, team training 
was more effective than individual training. Further research into this area 
should provide information which can impact on the issue of where and when to 
conduct team training. For example, individuals in air squadrons are trained 

within a team context. They are then individually assigned to operational 
units. If team skills are relevant to a given task situation and can be 
learned during a limited training period it may be more efficient to assign 
the team intact rather than disband it following training [41]. 
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A need exists for studies which explore the issue of skill training 
sequencing. After individual and team skill requirements for a given task 
situation have been idenitified, the sequence in which component skills are 
trained can be systematically varied and the resulting effects compared. 
For example, individual skills can be trained prior tc team skills training; 
training of team skills can occur first; or both individual and team skills 
can be integrated within the training program and taught concurrently. The 
results of such studies which can be performed in either established or 
emergent task situations will help determine when and under what conditions 
team training can be administered most effectively. 

Some types of team training are so costly that one cannot conduct them in 
an operational environment. Other types of training have logistic or safety 
factors which preclude operational, full fidelity training or testing. High 
fidelity simulations which were common in individual training for many years 
has been shown to be effective for team training. However, such simulation 
techniques, though costly, provide little generality beyond a specific training 
mission. A substantial research and development effort is needed to determine 
the degree of simulation fidelity required for effective training in different 
situations. Relatively low-fidelity simulation techniques have been effective 
and economical in individual training. However, the review of the literature 
indicated that high fidelity simulations were needed for effective team 
training. The studies surveyed were performed in established laboratory 
settings. The search should continue for effective low fidelity devices or 
methods to apply to team training in emergent task situations because of 
their potential cost savings. 
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The second question addressed in this review involved the identification 
of team training/evaluation techniques which warranted further study and 
support. One of the most promising new approaches is two-sided engagement 
training. Current implementations of this technique were described in 
Chapter IV in discussions of SCOPES, REALTRAIN, and MILES. The two-sided 
engagement simulation provides a low cost, high fidelity framework for 
developing individual skills. It provides a situation in which there is 
high motivation fostered by competition in a natural, realistic environment. 
This approach, which can be classified as situational or contextual training 
is an outgrowth of the effective "functional context" approach to individual 
training [78], Up to now, two-sided engagement training has been used to 
teach individual skills within a group context. Support is needed to study 
the feasibility of applying these methods to train team skills. There is a 
need Co study how effective the techniques are, analyze the factors which 
produce favorable training outcomes and study how these methods can be 
transferred to other training situations. Thus far, post-exercise debriefings 
have been used to discuss individual performances. Team performance infor- 
mation needs to be fed back to trainees in a more timely and effective manner 
both during and afttir training. 

A systematic investigation of feedback effects upon team training is needed. 
Support is required to do research on the fo lowing issues: 

• Need for feedback - \^at changes in training methods or content result 
from feedback? Will there be differences if remediation is contingent' 
or non-contingent upon feedback? 

• Feedback mode - Methods should be available for giving both trainees 
and trainer feedback of how well the tccim is performing. Determining 
what are the appropriate methods for giving performance feedback to 
team members is an area requiring further R&D (e.g.. Compare video 
recording with post-exercise debriefing procedures — Hypothesis: 
Clarity and objectivity of observation will provide more useful 
feedback) . The use of television and video recording to provide 
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relatively responsive performance feedback has been used successfully 
in a variety of contexts (e.g., sports, teacher training). Its potential 
value to military team training appears great, but needs considerable 
R&D support to identify its most appropriate applications. 

• Delay of feedback - A systematic study is needed to determine under 
what conditions is provision of feedback immediately after a training 
exercise, superior, equal to, or inferior when compared to delayed 
presentations — 1, 2, or more days following the event. 

Another issue which merits study concerns the effects of team member 
familiarity on the acquisition of team skills. Does the length of time that 
a team functions together affect team performance. One could systematically 
vary this time parameter, re-allocate team members, set up new teams, and 
then study transfer effects. 

The ability to evaluate oneself accurately is a skill which should help to 
produce effective team performance. Analysis of one's own errors, as well as 
awareness that one is overburdened and needs other team members' help are two 
specific components of this ability which should be trained. There is a need 
for a program supporting the development and evaluation of training; packages 
which would provide generalizable instruction in these self-assessment skills. 
One could compare subsequent team performance with or without prior self- 
assessment training. Greater proficiency would be expected from those teams 
who have had such self-assesf.ment skill training. 

Another possibility is that prior training of team members to assess the 
capabilities of others will result in better subsequent team performance. 
By having the ability to assess other team members' strengths and weaknesses, 
it is hypothesized that the individual team member best suited to take over 
or help out on a given activity can be selected adequately — and thereby 
enhance overall team performance. 

( 
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It may be more efficient to do research on a combination of some of thq 
issues discussed above within the same study. For example, groups feedback 
-^or skill sequencing issues can be studied in environments which contain 
established or emergent task situations and which vary in their degree of 
simulation fidelity. 

The research and development implications mentioned in the paragraphs 
above are merely the foci of what should be considered programmatic efforts 
in the team training/evaluation area. Considering the importance and extent 
of this area, the current state of knox^ledge is quite meager. Studies and 
techniques such as those mentioned above, if supported, should help to 
ameliorate this condition. 
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